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CLAIMS: We claim: 

1 . A method for enhancing facial images in a video by superimposing virtual object 
images onto said facial images automatically and dynamically in real-time, comprising 
the modules of: 

(a) initialization module; 

(b) facial feature detection module; and 

(c) superimposition module. 

2. The method according to claim 1 , wherein the (a) initialization module further 
comprises steps for: 

(a) capturing a plurality of images for an individual or a plurality of people with a 
single or a plurality of means for capturing images, 

(b) processing a single image or a plurality of images from said captured plurality of 
images in order to detect face for an initial face detection, 

(c) tracking said detected face in real-time, 

(d) verifying said face, and 

(e) estimating regions of interest for each facial features in said face. 

3. The method according to claim 1 . wherein the (b) facial feature detection module 
further comprises steps for: 

(a) combining a plurality of real-time facial feature detection approaches to get facial 

feature coordinate information from said face, 
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(b) block-processing of said facial feature coordinate information, 

(c) applying fusion algorithms and geometrical constraints to said facial feature 
coordinate infomnation. and 

(d) smoothing said facial feature coordinate information. 

4. The method according to claim 1 , wherein the (c) superimposition module further 
comprises steps for 

(a) preparing virtual object images, 

(b) validating said facial feature coordinate infomnation, 

(c) smoothing, 

(d) aesthetic processing, and 

(e) processing final super-imposition. 

5. The method according to claim 1 , wherein the method further comprises a step for 
processing touch-free interaction between said individual or said plurality of people and 
the system, 

whereby said touch-free interaction with said system enables said individual or said 
plurality of people to choose said prepared virtual object images on a means for 
displaying to superimpose onto said facial image. 

6. The method according to claim 2, wherein the method further comprises a step for 

combining the face detection process and said real-time face tracking process, 
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whereby the integration enables efficient and robust real-time facial image processing. 

7. The method according to claim 2, wherein the step for verifying said face further 
comprises a step for processing local face detection, 

whereby said verification makes correct facial images to be passed on to said facial 
feature detection module. 

8. The method according to claim 2, wherein the step for estimating regions of interest 
for each facial features further comprises a step for estimating said regions of interest 
dynamically, 

whereby said regions of interest change according to the results from said verification 
step, and 

whereby said regions of interest are used as boundaries for detecting each facial 
features, such as eyes, nose, and mouth on said detected face. 

9. The method according to claim 3, wherein the step for block-processing further 
comprises a step for applying said block-processing to each of said plurality of facial 
feature detection approaches. 

10. The method according to claim 3, wherein the step for smoothing said facial feature 
coordinates further comprises a step for storing a history of said facial feature 
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coordinates and applying a smoothing algorithm for the current facial feature coordinate 
using said history of said facial feature coordinates. 

1 1 . The method according to claim 4, wherein the step for preparing virtual object 
images further comprises a step for setting pivot points in said virtual object images. 

12. The method according to claim 4, wherein the step for aesthetic processing further 
comprises a step for processing rotation and translation of said virtual object images. 

13. An apparatus for enhancing facial images in a video by superimposing virtual object 
images onto the facial images automatically and dynamically in real-time, comprising: 

(a) means for capturing a plurality of images for an individual or a plurality of people, 

(b) means for processing said plurality of images, and 

(c) means for displaying output, 

wherein said means for processing said plurality of images further comprises means for 
detecting face and facial features in said plurality of images for said individual or said 
plurality of people, tracking said face images, superimposing virtual object images onto 
said face images, and processing user interaction. 

14. The apparatus of claim 13, wherein said means for capturing a plurality of images 
further comprises one or a plurality of camera and one or a plurality of frame grabber. 
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15. The apparatus of claim 13, wherein said means for capturing a plurality of Images 
further comprises means for controlling field of view dynamically. 

16. The apparatus of claim 13, wherein said means for capturing a plurality of images 
further comprises firewire or USB digital camera. 

17. The apparatus of claim 13, wherein said means for processing said plurality of 
images further comprises one or a plurality of processors perfonnlng the steps of: 

(a) detecting and tracking a plurality of face images from said plurality of captured 
images for said individual or said plurality of people, 

(b) verifying said face images within the face tracking window, 

(c) detecting facial features by combining a plurality of real-time facial feature 
detection approaches, 

(d) superimposing pre-defined virtual object images onto said face images using 
results from the facial feature detection, and 

(e) handling interaction between the user and the system. 

18. The apparatus of claim 17, wherein the apparatus for said step (b) verifying said 
face images within the face tracking window further comprises means for estimating 
regions of interest dynamically. 
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19. The apparatus of claim 17, wherein the apparatus for said step (c) detecting facial 
features by combining a plurality of real-time facial feature detection approaches further 
comprises means for applying fusion algorithms and geometrical constraints. 

20. The apparatus of claim 17, wherein the apparatus for said step (d) superimposing 
pre-defined virtual object images onto said face images using results from the facial 
feature detection further comprises means for preparing said virtual object images by 
setting pivot points in said virtual object images. 

21 . The apparatus of claim 17, wherein the apparatus for said step (d) superimposing 
pre-defined virtual object images onto said face images using results from the facial 
feature detection further comprises means for processing validation of the feature 
coordinates, smoothing of said feature coordinates, and processing aesthetic look of 
said virtual object images. 
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